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DISCLAIMER 

The reader is cautioned that computer programs developed in this research may 
not have been exercised for all cases of interest. While every effort has been made, 
within the time available, to ensure that the programs are free of computational and logic 
errors, they cannot be considered validated. Any application of these programs without 
additional verification is at the risk of the user. 

All icons images used in software for this thesis were either designed by the 
author or acquired as public imagery provided by Sun Microsystems™. Due to common 
industry-wide iconography, included images may bear resemblance to existing 
commercial graphics. No attempt has been made at infringing on any copyrighted 


imagery. 
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EXECUTIVE SUMMARY 

Joint Vision 2010 establishes a conceptual template for leveraging technological 
opportunities to achieve new levels of effectiveness in joint warfighting capabilities for 
the United States military. Of JV2010’s five areas of focus, Information Superiority is 
the most affected by emerging computer, communication, micro-miniaturization, and 
Internet technologies. Information Superiority will play a dominant role in the future of 
warfare. Technological advances in satellite imagery, remotely piloted vehicles, mobile 
communications, the Internet, and GPS alone have already swamped military leaders 
with more information than they can effectively use in a tactical situation. The ability to 
collect, analyze, and disseminate vast quantities of useful information is Joint Vision 
2010’s primary vehicle to achieve “Battlespace Dominance.” 

The Department of Defense has responded to Joint Vision 2010’s technological 
challenge with various advanced warfighting programs like the Command Post of the 
Future project, the Land Warnor 2000 project, and the Naval Postgraduate School 
Loosely Coupled Components Research Group. Al] three programs employ Internet-time 
technologies in wireless networking, command and control, and mission planning. 

How do military leaders know they are getting the biggest technological bang for 
their research dollar? How do they know if the newest techno-information system is 
overwhelming their commanders or giving them the Information Superionty they need to 
win the battle? The answer to these questions is based in Human Factors and the Human 
Computer Interface. Whether it is a laptop computer screen, a remote imaging device, or 
a computer aided rifle-sight, humans are interacting with a computer. No matter how 
powerful the interface appears, the sailor or soldier operating it must be considered in its 
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design. Technological advances are only moving in one direction — smaller, faster, and 
more complicated. Knowing that military research and development has _ fully 
incorporated the user’s needs into interface design ensures a useful, powerful, and 
effective information system. 

This thesis evaluates the human computer interface of existing map-based 
mission planning tools developed by a faculty and student research group in the Naval 
Postgraduate School’s Operations Research Department. After careful human factors 
based analysis, various usability flaws were identified and an improved Graphic User 
Interface was designed. The new interface, written in Java, utilizes any imagery that can 
be geo-referenced and will soon incorporate numerous operations analysis decision tools 
for the military planner. A usability study was conducted and results were compared 
with industry and the DoD standards for usable interfaces. The new map-based planning 
tool’s interface was determined to be usable and therefore was accepted as the base 
interface for further operations analysis decision-making tools. Not only is the new 
graphic user interface a DoD model for Java map-based mission planning software, but 
the Loosely Coupled Components research group is leading the way in military software 


development that incorporates human factors in the software design process. 
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I. INTRODUCTION 


The module is a scale of proportions that makes the bad difficult and the 
good easy. 


- Albert Einstein to Le Corbusier (1964) 
referring to intellectual model representation 


A. PROBLEM 

Joint Vision 2010 establishes a conceptual template for leveraging technological 
Opportunities to achieve new levels of effectiveness in joint warfighting for the United 
States military (CJCS, 1996). Of JV2010’s five areas of focus, Information Superiority is 
most affected by emerging computer, communication, micro-miniaturization, and internet 
technologies. 

The Information Superiority concept exploits advances in collection, processing, 
and dissemination technologies to “mitigate the impact of the friction and fog of war,” 
while at the same time, denying the enemy the nght to the same (CJCS, 1996). 
Information Superiority is achieved by fusing and processing information from all 
available intelligence sources and disseminating a usable product to thousands of 
locations in a timely manner. Using superior information, our joint fighting forces will 
achieve “Dominant Battlespace Awareness’ allowing increased force dispersion, 
mobility, and lethality (CJCS, 1996). 

The Department of Defense has responded to Joint Vision 2010’s Information 


Superiority call with several technology based research programs. Some examples are 


DARPA’s Command Post of the Future (CPOF), the Army’s Land Warrior 2000 program 
(LW2K), and the Naval Postgraduate School’s Loosely Coupled Components (LCC) 
research group. 

The LCC research combines several Operations Research based tools to aid in 
map based mission planning. Some of the tools are: National Imagery and Mapping 
Agency (NIMA) formatted map display, intelligence imagery rubber-sheeting, route- 
planning, shortest-path decision aids, whiteboard-style map annotations, battlefield 
training monitors, and field-deployed database servlets. Though each is an effective 
stand-alone tool, when fused into one interface they become ineffective and unusable 
beyond the academic environment. The fault in the software’s poor usability lies in its 
outdated software design process. In order to make the software more usable, the 
software design process must be improved and adapted to industry-wide Human 
Computer Interface (HCI) design standards. After examining the existing software under 
the new design process, it was determined by the LCC research group that the only 
acceptable approach was to design a new map-based mission-planning tool from the 
ground up. This thesis proposes a replacement to the existing LCC map-based mission- 
planning tool and developed a human factors software design methodology for future 


LCC Operations Research systems. 


B. HUMAN FACTORS AND GUI DESIGN 


Information Superiority based in technological advances is achieved when a 
human is able to successfully use a computer to collect, process, and disseminate 


information. Each DoD research group is developing its own graphic user interface 


to 


(GUI) to manage these processes. Their GUI’s will allow soldiers, sailors, airmen, 
marines, medics, commanders, or SEAL’s to interact with their computers to exchange 
information with any computer. Though technology and software electronically bridge 
gaps between dissimilar computer systems, process large quantities of information, and 
adapt to unknown hardware, the soldier is the part of the equation that puts the 
technology to use. If she or he is bogged down wrestling with menu structure or is 
unfamiliar with applications of his software, no technology can lead to success. A strong 
set of Human Computer Interface (HCI) guidelines is needed to ensure the soldier’s 


information display system has been designed with his tasks and needs in mind. 


1. Graphic User Interfaces 


The success of Microsoft Windows® and the Macintosh® O/S as popular Graphic 
User Interfaces is based on their designers’ strong incorporation of the Mental Models 
and Metaphors methods as discussed by Wickens, Gordon, and Liu (1998). The Mental 
Models method is a dynamic model of the user’s knowledge of the following: system 
components, how the system works, how components are related, what the internal 
processes are, and how the user affects the components. The Metaphor method is the 
process of using objects and events in a software system that are taken from a non- 
computer domain such as “desktops,” “cut and paste,” and “trash cans” (Wozny, 1989). 

Mayhew (1992) states that designers should enable the user to develop an 
effective Mental Model. An effective mental model is one in which the user can mentally 


represent the relationships between or perform actions on working components of the 


GUI. Wickens, Gordon & Liu (1998) provide four suggestions to improve the mental 
model, (1) Make invisible parts and computing visible to the user (1.e. dragging a file to a 
trash can to delete it), (2) Provide feedback to the user (i.e., showing statuses of loading, 
saving, printing), (3) Build in consistency (1.e., established patterns and rules common 
across applications), and (4) Present functionality through a familiar metaphor utilizing 
real world analogies (i.e., physically moving a mouse pointer through the non-physical 
environment of a computer display). 

The Metaphor Method, the second half of effective GUI design, provides the user 
with familiar metaphors for completing tasks. One example 1s the ability of World Wide 
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Web users to chat on the Internet in a “room.” Though none of the actions physically 
occur, users can identify with the metaphor of entering a room full of people and chatting 
with one or all si them. Other less obvious metaphors include matrix-structured 
spreadsheets, desktops, clocks, calendars, and back/forward icons for turning “pages” of 
virtual books, manuals, or Web pages. 

Interface designers must also be careful using metaphors that are also vulnerable 
to errors. Differences between the metaphorical world and the software system, if not 
made explicit, can cause errors or gaps in the user’s mental models of the software 
system (Halasz & Moran, 1982). Examples of metaphor error are turning pages left and 
nght in a virtual book by using Page Up / Page Down keys or pressing the MS® “Start” 
button to initiate a computer shutdown. 


GUI technology and design are not limited to conventional keyboards, monitors, 


and speakers. GUI design, based strongly on Mental Models and Metaphors, is crucial to 


implementing successful present and future software applications. “Thirty-seven to fifty 
percent of [industry] efforts throughout the software life cycle are related to the system’s 
user interface” (Hefley, Buie, Lynch, Muller, Hoecker, Carter, and Roth, 1994, p.315). 
Financial implications of these efforts force software development companies to join 
human factors engineers and software programmers at the onset of system desi on. Their 
goal 1s to reduce the short and long-term costs associated with poor design. As 
computers and displays become smaller, GUI’s will become ever more important and 
will be relied upon to maintain the information bandwidth required to complete 
complicated tasks. Task completion, however, is not the only yardstick for declaring a 
GUI design successful. The experiments in usability measure the quality and 
effectiveness of GUI designs. 


5 Usability 


The key to a successful GUI is whether or not it is designed with human usability 
as its primary goal. In the past decade, computer use has expanded to toddlers and the 
elderly and from making scientific calculations to writing e-mail and joining virtual 
combat missions. Software companies can no longer afford to push highly technical and 
unfriendly software on customers. To stay in business, they are meeting these widening 
demands through extensive usability research and adherence to HCI guidelines. 
According to Nielsen (1993), software usability is traditionally associated with 
learnability (ability to quickly become productive with software), efficiency (high 


productivity after initial learning period), memorability (relearning is not necessary after 


periods of non-use), errors (low error rate, ease of error recovery, no catastrophic errors), 
and satisfaction (subjectively determined). 

There are many ways to measure whether a Graphic User Interface meets the 
minimum requirements in an established set of standards. Each interface requires a 
different testing method. The most commonly used methods to measure usability are 
number of errors, time to perform tasks, and user subjective reactions. 


3: Usability Testing 


The overall goal of interface usability testing is to identify and rectify usability 
deficiencies in computer-based human computer interaction (Rubin, 1994). There are 
many measures of a GUI’s usability. Listed here are a few used by Microsoft’s® usability 
labs: benchmark studies, heuristic reviews, task analyses, error analyses, and competitive 
studies (Microsoft, 1998). Nielsen (1999) explains the simplest usability metric 1s 
success rate, which is the best estimate of the true success rate for a similar population 
user. In order to determine if a success rate 1s acceptable, standards must be established 
prior to usability testing. Either benchmarks from previous studies or heuristic industry 
standards provide the ruler for newly collected data (Nielsen and Molich, 1990, Nielsen 
1994). Industry standards for icon identification success rates average 70% for initial 
exposure and 100% there after (Bickford, personal communication, 11 May 2000). 
Standards for acceptable task analysis success rates are 90% (Bickford, personal 


communication, 11 May 2000). 


4. Human Computer Interaction Guidelines 


In order to make HCI successful across all platforms and software, interface 
guidelines must be thoroughly integrated into the product realization process (Lund & 
Tschirgi, 1991). But before HCI and the production processes can be integrated, HCI 
must be well understood. 

Dix, Finlay, Abowd & Beale (1998) define HCI by breaking it into three parts. 
The human user is any individual or group of users in an organization participating in 
task or process completion. The computer is any technology such as a palmtop, laptop, 
desktop, mainframe, or process control system. Interaction is any direct or indirect 
communication between a user and a computer to accomplish tasks. Since the early 
advent of Macintosh’s® desktop, Microsoft’s® Windows series, and the vast array of 
Internet browsers, human factors experts have been applying their expertise to HCI. 

It is also important to understand that common philosophical HCI guidelines 
serve aS a guide and base to all interface designers, no matter what company mandated 
standards are in place (Hix & Hartson, 1993). HCI guidelines are not limited to blue chip 
companies and Silicon Valley software developers, either; the European community has 
also recognized the requirement for common guidelines. 

In response to the need for common visual display terminals (VDT’s) in the 
European Banking and Economic Area, European Community members transformed a 
previously human factors related “minimum safety and health requirements directive” 


into national law. They have gone one step further by requiring software developers, 


who may lack knowledge in the area of human factors, to utilize design-aid software 
tools that incorporate built-in human factor guides and testing criteria (Reiterer, 1993). 
Even after subscribing to HCI guidelines, some experts believe that we have not 
evolved from our early non-GUI days of computing. Raskin (1997) explains that our 
present systems are as large, complex, and nightmarish as the mainframes they first 
displaced; he adds that to be a “power-user” one 1s expected to know, on average, over 


three hundred settings of the system he is using. 


© BACKGROUND 
Je LCC Software Design Processes 


The previous Loosely Coupled Components Software design process was 
simple. When a new idea for a map-based OR tool was discovered, it was immediately 
coded for proof-of-concept. The process allowed little or no human factors application 
until after the code was shown to work. The outcome of this process has been a sound 
OR tool that only a few people can utilize. Some of the user difficulties are discussed 


later in this chapter. Figure | shows the design process used prior to this thesis. 


Product 
Release 





Figure 1. Existing LCC Software Design Process 


The key to a successful and usable piece of software is incorporating 
human factors at the start of the design process. Modeled after Lim and Long’s (1992) 
Structured Human Factors Design Framework, figure 2 shows an adapted version for the 


LCC research group. 
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Figure 2. Improved Software Design Process 


Dis Current Information Display Systems Research 


The Defense Advanced Research Projects Agency (DARPA) is researching ways 
to provide an integrated display and analysis tool to military commanders to aid decision 
making abilities via the Command Post of the Future (CPOF) program. The Land 
Warnor 2000 project is investigating wirelessly networked wearable-computers and 
associated displays for use by individual soldiers to increase their combat effectiveness. 
The LCC project is developing methods to loosely connect various Commercial Off-The- 
Shelf (COTS) components to create military systems for mission planning and execution 


that contain the capabilities envisioned by Joint Vision 2010. 


a) DARPA’s Command Post of the Future 


DARPA’s CPOF project is to develop advanced technology to create an 
adaptive, decision-centered, visualization environment for the future commander with the 
end goal of doubling the speed and quality of command decisions while cutting the 
required support staff in half (Page, 2000). Page, project manager of the CPOF program, 
further states, “As current technology floods the military commander with messages, 
images, and data, he will require larger staffs and more computers to process, interpret, 
integrate, and understand the incoming information streams (Page, 2000, p.1).” 

Recognizing the human difficulty in processing large volumes of 
information at high rates, DARPA is incorporating various advances in Human Computer 
Interaction (HCI) technologies into its design. 

Some of these technologies include GUI-based 3D_ visualization, 


interactive 3D techniques, Natural Language processing, and Knowledge Base querying 


(Despain & Westervelt, 1997). They are also investigating human-computer interfaces 
that go beyond current GUI technology. Some examples include the creation of a “cyber- 
warmior,’ or computer-enhanced soldier, who could utilize visual cortex implants, 
vibration, temperature, eye-trackers, voice control, data gloves and intelligent user 


interfaces. 


b) U.S. Army’s Land Warrior 2000 


In 1991, an Army Science Board Study recommended that the soldier be 
treated as a “complete fighting system;” this recommendation resulted in the initiative 
known as the Soldier Integrated Protective Ensemble (SIPE). After proof of concept, the 
SIPE Program evolved into the Land Warrior (LW) program in July 1995 and has since 
become the Land Warnor 2000 (LW2K) program. The LW2K program integrates a 
computer and a soldier into one networked fighting system. By combining advances in 
computer and communications technology, inexpensive COTS hardware, and advanced 
weapons aiming systems, the Army plans to employ each networked soldier as a 
complete weapons platform. (Jette, 1999) 

The soldier will access his computer-enhanced system via the GUI on his 
handheld flat panel display or its near-future replacement, a helmet-mounted monocle. 
The Army plans to display digital messages, video, thermal site imagery, graphics, 
waming messages, and navigation information on either the monocle or the handheld 


panel. 


c) NPS’s Loosely Coupled Components 

The Naval Postgraduate School’s Operations Research Department began 
researching CPOF ideas and Land Warrior’s networking concepts in 1996 with an 
Operations Research look at decision aids, electronic cartography, and secunty. The 
primary goal of the LCC project is to design, develop, and demonstrate decision support 
systems for military planning, execution, and training using COTS technology from the 
fields of ae networking, Java°-based object-oriented programming, portable 
information display systems, war-fighting training systems, and mission planning tools 
(Bradley, Buss, & Shaw, 1998). These LCC objectives are achieved through a powerful 
object-oriented software concept to which new software modules and COTS hardware 
can be added and removed seamlessly via the Internet or wireless LAN. Some of the 
COTS equipment includes Palm IIIx™ PDA’s, Casseopias™, Libretto™ palmtops, bar- 
code readers, Lucent™ WAVELAN cards, and global positioning systems. 

The power of object-oriented programming using the Java programming 
language lies in platform independence and dynamic loading. Platform independence 
allows software to be “written once, [and] run anywhere” (Linden, 1997). | Dynamic 
loading allows even the least capable computer to conserve memory by downloading and 
running software only when needed and then purging it upon completion. 

Platform independence and dynamic loading are relatively new in military 
software design. Current military software design does not incorporate platform 
independence. It relies on a team of contracted software engineers who develop different 


versions of the same software tailored to meet the varying computing requirements of the 


military user. The Aegis computer software on U.S. Navy cruisers and destroyers best 
exemplifies this method of software design. As the ship’s combat computer hardware is 
upgraded in staggered fashion throughout the fleet, it requires a new software baseline 
version. These differences in software baselines can limit the interoperability of ships 
employing the Cooperative Engagement Capability (CEC), which shares fire-control 
information between ships for ballistic missile engagement. 

Dynamic loading is also an under-utilized concept in the military. The 
LW2K program will be the first to incorporate the ability for a minimally capable system 
to download programs, similar to Internet applets, as needed from network databases and 
then to purge them upon task completion. Examples of possible programs include 
optimal track routing, logistics calculators, and automated re-supply software. 

Combining the strengths of platform independence and dynamic loading 
via object-oriented programming is the first step to creating a powerful network of 
tactical military computers. The second step is a local area network. Both the LW2K 
and LCC project are investigating and utilizing current wireless network capabilities. 
The tactical benefits of wireless networks are high mobility, encrypted information flow, 
and seamless LAN-entry and -exit of portable systems. The following are just a few of 
the possible information flows the LCC project incorporates: updated unit positions, 
Current orders, and decision aids to acquire locations of nearest supply and medical 
stations. None of these capabilities can be utilized without a reliable wireless network 
connecting up to hundreds of portable computers and a robust visual interface to put the 


information at the soldier’s fingertips. 


Di EXISTING LCC SOFTWARE 

Loosely Coupled Components (LCC) software was developed through a series of 
Operations Research (OR) Master’s Theses (Bilyeu, 1998; Hattes, 1999: Schrepf, 1999). 
As each student developed a new OR tool, their module was attached to the existing LCC 
software through a common button bar. The overarching software hub is Thistle. 

TF Thistle 

Schrepf (1999) developed Thistle to simulate and model movement of ground 
forces which assists commanders make decisions for routing of convoys. The three 
pnmary GUI design flaws identified in Thistle are: poor menu structure and 
Iconongraphy, poor use of Screen Real Estate, lack of Positional Constancy. 

Schrepf’s interface was designed as a programmer’s device to incorporate various 
Java classes into one program. While it allowed users to initiate numerous Operations 
Research modules by pressing their associated buttons, it was not designed for the 


untrained. 


RQ THISTLE 5.0 Beta [cpm] 





[Fan] Convoy Briefing | Monitor | sorice | RouteBuilder | WALKER Observer | Graph | Intel Overtay | About | 


Figure 3. Thistle’s button bar continued to grow as new tools were added to the program. 


Every button has a text descnptor in place of a graphic icon. The human factor 
(HF) flaw in this design is two-fold. First, the user must read through the text of many of 
the buttons prior to finding the desired module. A better design would be to place the 
text in drop-down menus or use metaphorical icons. Second, the button bar is not 
organized by task or by commonality. Some of the modules use maps, even though there 
is a “Map” button on the bar. Once again, a multi-leveled drop-down menu would have 
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been a better choice by organizing related modules in the same menu structure (Minasi, 
1994). This menu should provide the basic structure of the software to the user without 
having to move a mouse or press a key. Thistle’s menu structure is not intuitive to the 
user. Without any previous knowledge of what some of these buttons do, a novice user is 
left to experiment with each button until the desired feature appears. 


If the user initiates several applications from Thistle’s button bar, the computer 


monitor display becomes crowded and confusing as shown in figure 4. 
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Figure 4 shows the second HF design flaw in the new software. Screen real 
estate, as many HF experts call it, is poorly managed in Thistle. Not only are there 
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overlapping windows, but between the non-overlapped ones lie large areas of wasted 
space. The largest cause for the misused real estate is the software’s ever-expanding 
interface design. Every new module opens its own window and piece of the user’s visual 
field. A single or doubled window design would have be a cleaner and less confusing 
way of displaying data to the user. 

The third interface design flaw was the lack of Recital Constancy. Positional 
constancy means the user can expect to find the same interface layout every time it is 
executed. Wickens, Gordon and Liu (1998) believe it is either through repetition and/or 
English language reading styles that users expect to begin any software application in the 
upper left-hand comer of the screen. Thistle utilizes multiple windows in various orders 
and positions that force the user to mentally track numerous module locations and 
eventually minimize and maximize module windows to locate them. The only result for 


novice users 1s frustration, confusion, and errors. 


Ze Flora 


The second primary module in Thistle is a dynamic map and overlay display tool 
named Flora that allows users to plan tactical missions, analyze networking problems, 
plan convoy routing and even monitor GPS-networked units. Several tools are available 
to manipulate the displayed maps and overlays. One tool was the zoom-in / zoom-out 
feature available in the pull down menu that replaces the active National Imagery and 
Mapping Agency (NIMA) map with the next larger or smaller scaled image of the same 


geographic location. A “Grease Pen” on-map annotation tool is available for planning 
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and analysis mark-ups. Users also have the option to load prepared overlays or create 
new ones with tools available in Flora. Overlays can consist of Grease Pen annotations, 
networking graphs with available networking algorithms, or unit symbology annotations 
for organic and multinational forces. 

Flora’s interface was the Loosely Couple Components’ (LCC) first real start at 
incorporating map-based mission-planning features into one GUI. Many of its GUI 
design flaws limit its ability to be widely used by non-expert users. Some of it design 
flaws are cumbersome zoom-in / zoom-out features, unclear button functions, poor map 
field of view and scope, and poor map re-centering or dragging. 


An example of Thistle’s mapping display Flora is shown in figure 5. 
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Figure 5. FLORA’s map display was the first tool to include map annotation tools. The 
“Grease Pen” button launches a window with some basic annotation tools. 


19 


S: SOFLCC 


The third major module in Thistle 1s SOFLCC which was developed to implement 
a platform independent mission planning and analysis system for the United States 
Special Operations Command (USSOCOM) (Bilyeu, 1998). SOFLCC combined the map 
display capabilities of Flora but added another unique feature - fading. The feature 
allowed the user to fade a map into an underlying satellite image that had been either geo- 
referenced at the same scale or “rubber-sheeted.” The term rubber-sheet involves 
acquiring recognizable landmarks on the map, the same landmarks in an image, and then 
through mathematical algorithms, stretch one or the other until they are synchronous. 
The map in figure 6 has been partially “Dissolved” into an underlying satellite image. 


Dark patches are forested areas corresponding to the contoured hills on the map. 
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Figure 6. Special Operations Forces Loosely Coupled Components added the feature of 
blending satellite imagery with maps as shown here. 
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As with Flora, SOFLCC’s design flaws lie in weak iconography and multiple 
windowing, and poor map manipulation. The user has to read the unevenly sized buttons 
until he finds the right one on the bar every time. A simple metaphorical graphic could 
increase the speed and accuracy of their use and improve user learnability. 

According to Nolan (1989), icons need to be concrete-familiar (non-confusing 
and common) as opposed to abstract and unfamiliar (confusing and uncommon). Text- 
filled tool-buttons in SOFLCC are concrete-unfamiliar. Novice users can read the 
function of the buttons, but may not inherently know how to use them to accomplish a 
task. 

Windowing again is a problem with SOFLCC. When the Show Overlays button 
is pressed, another window opens displaying all current overlays opened. And from that 
window, the user may choose from text-labeled buttons to “Hide Layers,” “Show 
Layers,” “Remove Layers,” or “Run Algorithm.” If “Run Algorithm” is pressed, another 
window opens to let the user load an algorithm. It is a complete surprise to a novice user 
that algorithms can even be run in SOFLCC or that they can be reached via the overlay 
buttons since there is no indication of such in the opening screen’s layout. 

SOFLCC fares even worse in its map manipulation. In short, there is none. The 
map itself is an overlay with no zoom capability at all. The user has no way of using the 
map for any purpose other than prepared overlays and algorithms. 


4. LCC Software Usability Summary 


Each piece of software was examined under the established guidelines set by the 


DoD HCI Style Guide (1994) and a variety of compilations by Schneiderman (1998), 
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Mayhew (1992), and Brown (1989). Due to time limitations, only Iconography and Map 
Manipulation Flaws will be addressed in the interface redesign. All of the Operations 
Analysis tools developed in both Flora and SOFLCC have proven necessity in military 


planning and operations and will be coded into the new interface as time permits. 


E. PURPOSE AND RATIONALE 


This thesis proposes a software design process and a GUI designed under widely 
accepted HCI guidelines, to replace the existing LCC software. It is not the author’s 
attempt to weaken the underlying power of any of the LCC programs, but strengthen 
them by folding them into an interface that 1s easy to learn, easy to use, and incorporates 
the features of all the programs. A usability study was conducted on the new GUI to 
determine its usability and to establish baseline data for comparison with future interface 
improvements. 

The new system, Mapping, Information, Display and Analysis Systems (MIDAS) 
exploits the same Java features the existing LCC software does, while maintaining 
networked map-based mission planning tools. Due to the vast changes in the interface 
design, a comparative study between MIDAS and the old interfaces was not feasible and 
deemed unnecessary. Therefore, the study’s scope was narrowed to the two major GUI 
improvements and their corresponding usability metrics: Iconography Recognition and 
Map Manipulation. The usability study measured learnability, memorability, and 
efficiency. It was hypothesized that MIDAS graphical user interface would be superior 
to the existing LCC graphical user interface software due to the adherence to human 


factors pmnciples. Specifically, MIDAS incorporated drop-down menus and 


metaphorical icons to aid user readability, standardized mouse functions across all map- 


manipulating tools, and managed its screen real estate in a clear and simple manner. 
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I. METHOD 


A. MIDAS 

Mapping, Information, Display and Analysis System is a software concept in re- 
designing existing LCC user interfaces for military mission planning utilizing the new 
LCC Software Design Processes shown in figure 2 and Appendix A. As a Java based 
program, MIDAS takes advantage of its object-oriented programming by incorporating 
existing Java classes from both Thistle/Flora and SOFLCC and leaving itself open-ended 
to the capability of importing new methods and modules via networks or the Internet. 
The largest contribution MIDAS should make to the LCC project is improving the 
usability of the map-based mission planning. MIDAS specifically addresses the three 
major flaws in Thistle/Flora and SOFLCC. 


i Iconography 


The icons chosen for MIDAS follow accepted human factors guidelines for 
familiarity, visual and conceptual distinctness, design detail, and consistency in scheme 
(Mayhew, 1992; DISA 1994; DoD 1999). The images were chosen to help the user in 
maintaining context and onentation while reducing the requirements for memorizing 
commands and syntax (Brown, 1989). The icons for each button are displayed in table 1 


with their associated functions. 


25 


Restores mouse to default features 





Enables the mouse to “grab” the map and move 
it around the screen to re-center or adjust 
_ the new 1mage’s center 

| Masnity= Zooms out from image with the “click” location 
as the new 1mage’s center 

a Brings up text entry box to place text at the 
location of the mouse “click” 

A left “click” will merge the top image into the 
























back image — anight “click,” the opposite 


Nia Draws a line from a “click” and “drag” to a new 
point 
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eS centered ellipse if the Shift key is held 

Draws corner-anchored rectangle or “click” — 
at centered rectangle if the Shift key is held 
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Clears all overlay objects from image 
Removes a single overlay from the image 
Restores a single overlay to the image 


Table 1. MIDAS’s buttons and their related functions 
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2, Multiple Widowing 

Unlike Thistle/Flora and SOFLCC, MIDAS displays one resident window. 
Temporary windows include a file chooser and color chooser. The user may reposition 
and resize the GUI to fit the screen as necessary. MIDAS departs from Thistle in one 
other major feature — expandability. In Thistle, new OR concepts were added to the 
program by adding a new window and a new button on the command bar. As future 
features are added to MIDAS, they will only assume “real-estate” required for the 
feature’s name on the menu bar. The feature’s pull-down menu will cascade into the 
screen like other menus and disappear upon completion of a specified task. No additional 
resident windows will be generated by any menu item as in Thistle. Figure 7 is a screen 


shot of the opening view of MIDAS. 
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Figure 7. MIDAS starts with most map-base tools visible to the user. No further menus 
are required to begin work on a map. 
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3: Map Manipulation 


Successful map manipulation in map-based systems relies on four basic points: 


image format, map movement, zoom, and map annotation. 


a) Image Format (.gri) 

The LCC concept requires MIDAS to be as flexible as possible with image 
formats and mixing thereof. Where some map-based systems require consistent use of 
one image type (i.e. .bmp, jpeg, .gif, .tif), MIDAS’ image handling started at ground zero. 
The two characteristics of any map required for successful use are the image itself and at 
least two known coordinates on the image. By combining the two into a geo-referenced 
image, the result is useful for all map-based planning. MIDAS, from the outset, only 
accepts geo-referenced images (.gri) formatted files (Buss, 2000). This new format 
solves the problem of using any referenced or non-referenced image type (satellite, 
hyper-spectral, intelligence, topographic, radar, remotely piloted vehicle (RPV), hand- 
drawn, etc.) in MIDAS. Files with the .gn format are created by a Java class that imports 
an image, takes latitude and longitude coordinates from the user, and senalizes them into 
an image with associated position tags. Upon importing or loading the image into 
MIDAS, the position tags are read which then transforms any pixel in the image to a 
latitude and longitude. Any annotations then made to any image are ported and resized, 
by latitude and longitude, to any other geo-referenced image that may overlap the same 


geographical area. This image format also has exciting possibilities in “rubber-sheeting” 
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non-referenced intelligence images to known geo-referenced images providing mission 


planners another tool in analyzing images. 


b) Map Movement 

Map movement is handled by two basic actions. The first is instantiated 
by a nght click of the mouse somewhere on the image. The result is re-centering the 
image at the location of the click. 

The second map movement is instantiated by clicking the “Grab” icon 
which enables the mouse controlled drag mode. When the left mouse button is pressed in 
this mode, the image can be dragged by movement of the mouse and repositioned by 


releasing the mouse button. 


Cc) Zoom 


Two buttons — the Zoom-In and Zoom-Out, control MIDAS’ zoom 
feature. As one can infer, the Zoom-In button replaces the current image with one of 
larger scale in the GUI pane. Zoom-Out completes the opposite function. To zoom in or 
Out on an image, the user first mouse-clicks on either zoom-in or zoom-out icon and then 
a location on the image. The place selected on the image becomes the center of the new 
display. For the usability study, the zoom levels were limited to four scales. The four 
zoom levels are shown in figure 8. Other methods for improved zooming will be covered 


in the discussion chapter. 
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Figure 8. In its first version, MIDAS’s zoomed images scale as determined by linking 
variously scaled maps of the same geographic area. 


d) Map Annotation 


For any map-based planning software to be effective, users must have the 


ability to annotate it. Thistle gave the user a “Grease Pen” tool allowing lines, circles, 


squares and symbols to be overlaid on the image. The same capability was incorporated 


into MIDAS in an improved fashion. Accompanying the mentioned functions, are three 


new tools: 


“undo,” “restore,” and “erase.” 


These tools allow the user to make 


corrections to map annotations. User-added annotations are also geo-referenced to the 


image allowing lines and marks to overflow to new images of the same geographic area 


and be resized to match the effects of zoomed images. 
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B. USABILITY EXPERIMENT 


i Participants 


Twenty participants volunteered for the usability study. Table 2 shows the 


breakdown of participants in the study. 


User Type oll ee USMC Civilian | Foreign | Totals 
ca Nav 
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Table 2. Usability Study Subject Demographics 
Sixty percent of the subjects reported previous use of map-based software. All 
had a neutral or positive attitude toward computer use. Average computer sessions per 
week were 15.2 with an average session length of 57 minutes. 


Figures 9, 10, and 11 show computer-use demographics for subject sample. 
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Figure 9. Participant’s software application usage 
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Figure 10. Computer Use Per Week (n=20) 
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Figure 11. Years of Computer Experience (n 
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ap Apparatus 


Subjects were seated in front of a standard 19” (17.75” viewable) computer 
monitor set at 65,636 colors, 1024x768 pixels, 85Hz refresh rate, and font size small. 
The processor was a Pentium III 500 MHz with 128 MB RAM and equipped with a 104- 
key standard keyboard and an Intellimouse®1.1A PS/2. 

A high-grade video recorder with audio was used to collect a record of each test 
session for additional information gathering on each subject. The video feed was 
broadcast to a High Definition Television to enable continuous visual test monitoring. 

3: Software 

MIDAS was written in Java 1.2.2 (Sun Microsystems, 2000) utilizing Borland® 
JBuilder3° and various text editors. 

Icon images utilized by MIDAS were bitmaps produced by the author using 
Microsoft’s® Paint® program and converted to Graphical Interchange File® (.gif) format 
using Microsoft’s® Image Composer®. 

4. Industry HCI Benchmarks 

Industry benchmarks for usability vary between companies and applications; 
therefore it is difficult to find published success rates for task completion, icon 
identification, and memorability. In order to establish a qualified heuristic benchmark, 
several experts in the HF industry were consulted for their opinion as applicable to 
MIDAS and its usability study. The following personnel provided a consensus of 
acceptable usability benchmarks to compare MIDAS usability test data. Jose Arcellena ts 


a Human Interface Specialist for the National Broadcasting Company Intemet Inc. Peter 
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Bickford is CEO of Human Computing Consulting Firm and former HF specialist at Sun 
Microsystems. Dr. Mary Cwerwinski is the head of the Adaptive Systems Interaction 
Group at Microsoft. Donald Gentner is Senior Staff Engineer and Human Interface 
Designer at JavaSoft, Sun Microsystems. John Pane is a Graduate Assistant to the 
Computer Science Department, Carnegie Mellon University. 

2: Procedure 

Subjects were asked to read and sign various consent and experiment information 
forms as well as a background questionnaire (Appendix B) based on Rubin’s 
questionnaire for computer usability studies (1994). Data gathered during the experiment 
was annotated on the Data Collection Sheet found in Appendix C. 

After all forms were completed, subjects were moved to a half-walled cubicle 
where the software was already running and seated at the computer station. They were 
then asked to describe the function, by iconography only, of selected buttons annotated in 
Appendix C. Subjects were then asked to enable the tool tips option and review the 
function of any button they were unsure of. Subjects were then asked to complete nine 
more pre-determined tasks also shown in Appendix C. Each of the tasks addressed the 
user’s ability to complete map-manipulating functions with no prior training or exposure 
to the interface. 

Each experiment session lasted approximately 15 minutes. Upon completion of 
the tasks, subjects were allowed to ask questions to clanfy any difficulties and make 


comments regarding the program’s interface or future use. 
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To test for Memorability of MIDAS’ GUI, ten subjects received an icon 
recognition test (Appendix C) approximately one week after their initial testing. Subjects 
were asked to provide the function of the same buttons that were tested in the initial test. 

User comments and recommendations for improved icons were annotated and are 


discussed in the Results. 
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II. RESULTS 


A. LEARNABILITY 

Learability was tested using two methods. The first was straight identification 
rates compared to industry standards. The second method determined that MIDAS’s icon 
identification rates are predictable. 

1. Identification Rates 

The industry standard for icon identification rates is 70% for initial contact 
(Czerwinski, personal communication, 11 May 2000, Bickford, personal communication, 
11 May 2000). This standard was used to establish which icons in MIDAS’ GUI required 
redesign. Figure 12 shows the icon identification rates for all twenty subjects across all 


thirteen icons. 
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Figure 12. Icon Identification Rates (n=20) 


ST 


The four icons identified as having poor learnability are Text, Erase, Undo and 
Restore, each of which scored below the 70% identification rate. The most common icon 


misidentifications or responses for the four icons are summarized in Table 3. 










| 


Table 3. Misidentification Responses and Associated Rates 
The icons’ poor learnability may be attributed to two reasons. First, they did not 
provide enough visual cues to the user to establish a metaphor for their use. Second, they 
matched too closely to icons in other software that perform very different functions. 
Third, they did not match icons from other software that perform the intended functions. 
The four poor icons should be redesigned and re-tested utilizing the study’s feedback and 
compared to benchmarks established. 


2: Icon Identification Predictability 


Leamability was also examined to determine if subjects’ scores were dependent 
upon his/her demographics. A least squares regression model was calculated to 
determine if subjects’ scores for icon identification were influenced by demographics 
(Agresti, 1990, Cook & Weisberg, 1999). Subject’s scores were determined to be 
dependent upon demographic data via the following statistically significant model with 
an alpha level of .05 (F (3,15) = 4.03, p = .0275). These results show that icon 
identification for a similarly demographic sample can be predicted by knowing the 
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average length of their computer session, how many sessions a week, and number of 


years Computer experience. 
Score = 4.6410+ (.0173)x (Length) — (.6045)x (Session) + (.5930)(Years) 


The regression model was validated using the metrics and test statistics shown in 


Table 4. Supporting graphs are in Appendix E. 
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Table 4. Regression Statistics 
These results are important for two reasons. First, they show future interface 
designers the demographics that determined successful identification in this sample 
population. Second, they provide the designer guidance to tailor the interface for a target 
population or take steps to broaden the identification rate across a larger demographically 


diverse sample. 
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B. MEMORABILITY 

The metric for determining Memorability was a second icon identification test 
one week after the initial exposure to the icons. The acceptable industry success rate for 
Memorability is 100% (Gentner, personal communication, 12 May 2000, Arcellena, 


personal communication, 12 May 2000, Pane, personal communication 12 May 2000). 
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Figure 13. Icon identification success rate for Memorability 


The majority of MIDAS’ icons scored well in Memorability. After only one 
exposure to the icons, 10 of 13 icon recognition rates were 100% when tested one week 
later. Icons with problems, Erase, Undo, and Restore, were well below the standard of 
100%. Their lower rates are due to the same reasons they had poor Learnability. The 
icons were confused with icons already learned from other software that complete very 
different functions than those in MIDAS. After three new icons are shown with 


acceptable Learnability, future studies should test shorter and longer in-between-use 
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times. These results show the majority of images selected for the button bar are easy to 


remember even without consistent use. 


C; EFFICIENCY 


Overall, MIDAS has an efficient interface. With no prior training, subjects are 
able to quickly become productive scoring above 90% for all map manipulation tasks. 


Figure 14 shows success rates for the ten map manipulation tasks described in Appendix 
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Figure 14. Map Manipulation Task Success Rates (n=20) 


Though users could not identify the function of all of the icons with the tool tips 
feature disabled, they were able to successfully complete the tasks with tool tips enabled. 


This result shows the success of the interface design concept that gives the user the 


4] 


ability to add helpful information to the interface to improve efficiency. The ability to 

turn the tool tips off also allows experienced users eliminate possibly annoying clutter. 
Future work in Efficiency should focus on an in-depth task analysis or scenario 

driven task list. Many subjects did request to use the software for personal or school- 


related work — a subjective sign that the interface was easy to learn and use. 
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IV. DISCUSSION 


The goals defined at the onset of this thesis were to produce a quality easy-to-use 
graphic user interface (GUI) for map-based mission planning and conduct a usability test 
to determine its design success. The usability study identified some weaknesses in icon 
design that in the next version of MIDAS will be improved and re-tested. The weakness 
were not strong enough to adversely affect the overall success of MIDAS’s interface. 

MIDAS successfully combines many of the proven tools from existing Loosely 
Coupled Components software into one streamlined design while incorporating the strong 
design points of established human factors guidelines. With continued GUI improvement 
and testing, MIDAS will grow to become a powerful and portable map-based mission- 
planning tool. Recommended improvements in MIDAS’ interface can now be tested 


against the benchmark established in this thesis. 


A. RECOMMENDATIONS 


MIDAS will continue to evolve under the guidance of the LCC research group. 
As new map-based Operations Research tools are developed, they must be subjected to 
the new Software Design Process and must comply with established DoD (1999), DISA 
(1994) and Industry HCI standards. Technology will continue to evolve bringing smaller 
displays and unique pointing devices to the doorstep of LCC. The research group must 
maintain an understanding of the human factors involved to successfully exploit the 


capabilities of these emerging technologies. 
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APPENDIX A. LCC SOFTWARE DESIGN PROCESS 
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APPENDIX B. MIDAS SUBJECT QUESTIONNAIRE 


1. What is your age? (20-25) (25-30) (30-35) (>35) 
2. Male or Female? M F 

3. Occupation? 

4. If military, what rank and branch? 

5. Highest Grade Completed? 


12 Assoc. BA/BS MA/MS MD/Ph.D. 


6. Which is your dominate hand? Left Right 


7. Are you currently experiencing any problems that impair your ability to use a 
computer? 


a) Yes b) No 
If yes, what are they? 
8. How many times do you use a computer a week? 1-5 = §-10—s: 10-15 
9. What is your most common computing session length? 

<10min 10-30min 30-60 min 60-90min >90min 

10. How many sessions of this type do you have a day? 

a) l 

Diez 

c) 3 


d) 4 
e) >4 
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>15 


11. Which of the following applications do you most often use on a daily basis? (circle as 
many as necessary) 


a) Send/Receive e-mail 
b) Surf the Internet 

c) Word Processing 

d) Finances 

e) Spreadsheets 

f) Games 

g) Presentations 

h) Programming 

1) Other 


12. What operating system do you primarily use? (circle more than one if needed) 


a) Windows 9X, NT, 2K 
b) Mac 
@) mAemtx 
d) Unix 


13. How many years have you been actively using a computer? 
a) <] 
b) 1-3 
c) 3-5 
d) 5-9 
e) >10 yrs. 


14. Have you used map-based software? (commercial, military, Internet, etc.) 


a) Yes 
b) No 


15. Are you geographically familiar with the Monterey Peninsula? 


a) Yes 
b) No 


16. What is your attitude toward computer use? 
a) Positive 


b) Indifferent 
c) Negative 
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APPENDIX C. MIDAS DATA COLLECTION SHEET 


PART |: ICONOGRAPHY 


“What do you think the functions of the following icons are? 
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PART 2: MAP MANIPULATION 
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Readout Herman Hall?” 
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APPENDIX D. FOLLOW-UP ICON RECOGNITION TEST 


MIDAS v.1 Usability Test Data Collection Sheet 
Part Two 


In two or three words, what do you think the functions of the following icons are? 
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APPENDIX E. REGRESSION SUPPORT 


Score = 4.6410 + (.0173)x (Length )- (.6045)x (Session) + (.5930)(Years) 


Residuals 





Quantiles of Standard Normal 


Score 





Fitted : Length + Sess + Years 
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